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(54) Title: INTEGRATED MULTIMEDIA INFORMATION MANAGEMENT SYSTEM 




(57) Abstract 

A multimedia management system, comprising a plurality of host computer platforms (9) and operating system software therefor, 
and video display screens (13-1, 13-2) and means for connecting the screens to the host computer platforms (9) to provide easy access to 
a plurality of media sources, including video, audio, and communication sources. In order to enable facile access to the media sources, a 
pointing device provides horizontal and vertical coordinates to a selected computer platform, and a controllable interface system couples the 
media sources to a selected host computer platform. A large screen display (14) can be coupled to the same information as displayed on 
one of the video display screens. The controllable interface system includes a programmable microprocessor and a bus system, a plurality 
of RS-232 interface cards (24, 25-1 through 25-5, 26 and 27) connected to the bus system. Video (20) and audio (19) switching matrices 
selectively couple the plurality of media sources to a selected host computer and two or more display screens are positioned for common 
viewing by a user and the pointing device (10-1) is a touch panel device. Speakers (18) and a data entry device for recording responses 
are also included. 
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Integrated Multimedia Information Management System 

I. Background of the Invention 

1. Field of the Invention. This invention relates in general to 
multimedia systems, and in particular to the integration of computers, computer 
software, video electronics, audio electronics, and communications electronics. 

2. Description of Related Art. If a political leader, business 
executive, manager, or teacher could retrieve, organize, query and present all 
known forms of information through a single integrated system, the impact on 
their efficiency would be astronomical. The ability to make informed decisions 
and to decisively influence others depends directly upon the availability of 
accurate and timely information. The ability to teach, inspire, and to enlighten 
others depends as much upon the information as the form in which it is presented. 
Finally, the strength of the economy and the ability to conduct commerce is 
directly related to the rate at which information and ideas flow, an issue as 
critical as the means by which manufactured products are transported. There 
exists several technologies, when properly merged, that can solve existing 
information management problems. 

The present invention is directed as a system and method of integration of 
existing technologies that will allow technically unskilled individuals to access, 
manipulate, and present multimedia information in a fashion that dramatically 
increases the rate of information flow to its final destination while decreasing the 
consumption of resources. 
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Numerous corporations have focused their efforts on a new technology 
presently called "multimedia". Multimedia is the integration of text, graphics, 
animation, video, and audio through the use of a computer. These companies 
have invested heavily to prove that multimedia will dramatically increase the rate 
of information flow, increase efficiency, reduce overhead, increase retention 
rates during training, and reduce the consumption of natural resources. "A 
picture is worth a thousand words" and when combined with text and audio 
increases the rate at which an individual can assimilate information. This 
combined with interactivity, allowing the user to navigate through information in 
a fashion more natural to the human mind, is the next generation of information 
management. However, with such a large number of independent efforts to 
mature this new technology, the vast majority of busy professionals that know 
they need the technology have found it impossible to find an easy to use turnkey 
solution. 

To pick a single company's multimedia product means picking from a 
limited number of media formats. It also means dependence upon a skilled staff 
to operate the system that is purchased. What exists today is a proliferation of 
standards that are mutually exclusive. A video expert deals with storage formats 
like VHS, S-VHS, Hi8mm, Super Hi8mm, Betcam, D2, 8" laserdiscs, 12" 
laserdiscs (CLV or CAV) and several other formats. Video can be routed in an 
NTSC, PAL, Y-C, RGB, or HD format. One must deal with television standards 
like NTSC, PAL, and HD and concerns over frequency ranges like VHF, UHF, 
and numerous cable channel frequencies. Computer video standards include 
EGA, VGA, SVGA, XGA, Mac, and many high resolution formats. Most video 
projection technologies or large screen displays such as flat Plasma Display 
Panels (PDP), etc... only support a limited number of these formats. Audio 
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comes in the form of cassette tape, DAT, CD, and numerous digital file formats. 
Audio is transmitted using various modulation techniques to include AM and FM 
at various frequencies. , The proliferation of networking electrical standards like 
LocalTalk, Ethernet, TokenRing, and FDDI as well as numerous protocols and 
file servers, forces a user to make more choices that further isolates them from 
some day tapping into a national information highway. Computer operating 
systems like DOS, Windows, Windows NT, OS/2, Macintosh System 7.1, Unix, X 
Windows, and Next all create information islands. The data storage, file format, 
MODEM, FAX, and optical disc technologies are also so numerous that to pick 
any single format increases the "Tower of Babel", unregulated, incompatible 
information maze that grows on a daily basis in this country. The present 
invention makes it possible to become independent of these compatibility issues. 

II. Summary of the Invention 

The principal object of the present invention is to provide a system of 
integrating existing computer, video, audio, and communications technologies. In 
a preferred embodiment this integration will take the form of a multimedia 
information management station that allows a user to manage multimedia 
information in a fashion that requires no programming, no scripting, no menu 
bars, and no exhaustive training. 

Another principal object of the present invention is to eliminate the 
dependence upon a single variant of the following formats, standards, and 
technologies in comprising the multimedia information management station: 
a. Computer platform (Apple®, IBM® and Compatibles, Sun®, 
etc.). 
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b. Computer operating system (Macintosh®, Windows®, OS/2®, 
Unix®, etc.). 

c. Video format. 

d. Audio format. 

e. Optical disc format 

f. Networking standard. 

g. Wireless communications standard. 

h. Software application file format. 

i. Video display technology. 

j. Video projection technology, 

k. Computer based pointing device. 

It is also an object of the present invention is to provide support for the 
control of the below listed components and processes through a single computer 
"pointing device". The "pointing device" being in the form of, and not limited to, 
a touch screen, mouse, track ball, digitizing pad, infrared remote, or radio 
frequency remote, all of which pass horizontal and vertical coordinates to a host 
computer. 

a. Read only and writeable video laserdisc players. 

b. Read only and writeable CD or CD-ROM players. 

c. Video cassette or tape recorders. 

d. Audio cassette or tape recorders, 
e Video cameras. 

f. Microphones. 

g. Audio mixers. 

h. Audio speakers (volume control). 

i. Video projection systems. 
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j- 




K. 


Facsimile machines. 


L 


Document scanners. 


m. 


Slide scanners. 


n. 


Video digitizers and manipulation of digitized video. 


0. 


Audio digitizers and manipulation of digitized audio. 


r\ 


^ompuier networKs. 


r\ 

q- 


viaeo encoders. 


r . 


Video switching matrices. 


s. 


Audio switching matrices. 


t. 


Room lighting. 


u. 


Room temperature. 


v. 


Cable TV selection. 


w. 


Televideo conferencing. 


X. 


Satellite communications. 


y- 


Radio frequency communications. 


z. 


Multimedia station ergonomics. 



A further object of the present invention is to provide a means to present 
multimedia information in a non-linear fashion supporting a level of interactivity 
that allows both the presenter/instructor and audience/student to search for and 
assimilate information in a more intuitive and natural fashion. 

The foregoing objects are accomplished by the proper integration of the 
following components: 

a. Control System - The control system includes a microprocessor 
with a bus architecture that supports control of a wide variety of cards to include 
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RS-232, Infrared In/Out, Volume Control, Relays, MIDI, SMPTE time code, 
voltage ramping etc... The microprocessor must be programmable and support 
at minimum an 8 bit data bus. Many manufactures offer solutions with various 
processors and card options which are suitable for the requirement of this 
component of the invention. 

b. Computer(s) - The computer can be of any manufacturer with 
any operating system that preferably supports the following: 

(1) 32 Bit Bus architecture (16 bit if cost is more important 
than performance). 

(2) 40 MBytes or more of RAM (less if cost is a factor). 

(3) Hard disk drive with preferably a minimum 200 MByte 
capacity. 

(4) CD-ROM with 300 KByte/second transfer rate (w/CD 
support). 

(5) Support for two 24 Bit color video displays at 640x480 
resolution. 

(6) Video digitizing at 30 frames per second. 

(7) Audio digitizing at up to 44.1 KHz, 1 6 bits per sample. 

(8) Two serial communications ports. 

(9) Pointing device. 

(10) Keyboard, pen based, or voice recognition text entry. 

(11) SCSI bus port. 

(12) Stereo audio out port. 

(13) Graphical user interface software library. 

(14) Support of LocalTalk, Ethernet, TokenRing, or FDDI. 

c. Video Laserdisc Player/Recorder. 

d. Video Cassette Recorder. 
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e. Speaker/Amplifier combination. 

f . Video Encoder. 

g. Video switching matrix. 

h. Audio switching matrix. 

i. TV Tuner. 

j. Connection patch panel (internal/external source connections), 
k. Optional miscellaneous components as required to include the 

following: 

(1) Audio cassette or tape recorders. 

(2) Video cameras. 

(3) Microphones. 

(4) Audio mixers. 

(5) Video projection systems. 

(6) MODEMS. 

(7) Facsimile machines. 

(8) Document scanners. 

(9) Slide scanners. 

In the preferred embodiment these components are integrated into a single 
wheeled cabinet that allows the system to be transportable. Preferably, the 
cabinet top supports two video display screens (with touch screen capability), 
keyboard, and trackball. One display screen is dedicated to viewing the 
"presentation" or displaying exactly what is projected if the output is connected to 
a video projector, large screen video monitor, or flat PDP displays through the 
supplied connection patch panel. The other video display screen is a virtual 
interface or control display screen that controls all components integrated within 
the cabinet. During a presentation, the user can randomly select any portion of a 
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multimedia presentation from the control display screen allowing the presentation 
to be driven by feedback from the audience. This allows the presentation to be 
non-linear. During the creation of a multimedia presentation, or multimedia 
training material, the two video display screens are used as virtual interfaces for 
manipulating the multimedia information that comprises the final presentation. 

The cabinet houses the control system, the video switching matrix, audio 
switching matrix, the user's choice of video devices, the user's choice of audio 
devices, and the users choice of computer platforms. The preferred embodiment 
includes an Apple Macintosh using a Motorola 68040 microprocessor (or better) 
and an IBM compatible using an Intel 80486 microprocessor (or better). Sun, 
Silicon Graphics, and other work stations could be used. Between these two 
platforms, the Macintosh OS, DOS, Windows, Windows NT, OS/2, and Unix 
operating systems can be supported, giving the user access to the largest possible 
spectrum of computer based information. The entire system is integrated to the 
point that the user need only plug in a retractable power cord and depress the 
single power switch to begin using the system. 

The software for the Control System resides in the Control System's 
memory and manages all requests from the computers to adjust volume, control a 
video device, switch the video matrix, dim the lights, close a curtain, etc... The 
Control System also supports a large quantity of digital alphanumeric touch pads 
or data entry devices which would allow students or members of an audience to 
pass data to the software executing on the computers. This supports significant 
interactivity between the presenter and their audience. The software that resides 
on the computers allows a pointing device to control volume, create multimedia 
presentations, play video tapes, run a televideo conferencing session, create a 
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commercial for cable TV, give a test to students, etc... The interface chosen for 
the software that resides on the computers has been developed to ensure a limited 
amount of complexity is present at any time, actions are intuitive, and the user 
need not understand programming, scripting, or the concept of menu based 
commands in order to make use of the system. 

HI. Brief Descrintion of the Drawings 

The above and other objects advantages and features of the invention will 
become more apparent when considered with the following specification and 
accompanying drawings wherein: 

Figure 1-A is a diagrammatic illustration of a preferred embodiment of 
the invention showing the multimedia management system controlling what is 
projected onto a large screen display with the seating of the audience, 

Figure 1-B is a diagrammatic illustration of a user's perspective of the 
preferred embodiment of the invention showing two touch screen video displays, 
the keyboard, and a trackball or pointing device, 

Figure 1-C is a diagrammatic illustration of a user's perspective of the 
preferred embodiment showing a user controlling the system via the touch screen 
video display, 

Figure 1-D is diagrammatic view showing an audience perspective of the 
preferred embodiment showing the stereo speakers, 

Figure 2 is a block diagram of a multimedia information management 
system incorporating the invention, 

Figure 3 - 7 are explanatory block diagrams, Figure 3 showing the 
composition of the control system and the paths of control for computer video 
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signals; Figure 4 illustrates the signal paths of the NTSC video out routing; 
Figure 5 illustrates the signal paths of the video in routing; Figure 6 illustrates 
the signal paths of the stereo audio out routing; Figure 7 illustrates the signal 
paths of the stereo audio in routing, 

Figure 8 is a view showing a typical presentation to a user on the virtual 
interface or control screen with a touch entry device and associated control icons 
displayed thereon for controlling all components integrated in the system, 

Figure 9 is a view showing the typical presentation menu according to the 
invention, 

Figure 10 is a view showing a typical slide menu according to the 
invention, and 

Figure 11 is a view showing the typical edit menu according to the 
invention. 

IV. Detailed Description 

System Overview. Referring to Figures 1-A through 1-D, cabinet 1 is 
preferably provided with wheels W, a podium configuration on top of the cabinet 
supporting left and right touch capable video display screens 4L and 4R, 
keyboard 5 and a trackball 6. A compact video camera (not shown) can be 
mounted by a flexible goose neck support post (not shown) to provide live 
information for display on screen 2 via the user's choice of video projection 
system (not shown). The left 4L or right 4R touch capable video display screens 
can be used as the virtual interface or control display screen, while the other 
screen lets the user see what is being projected to the audience . 
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Cabinet 1 houses the control system 9, computer platforms 10-1 and 10-2, 
audio and video switching matrices 19 and 20, the users choice of video and audio 
devices, video A/B switches 11-1, 11-2, and 11-3, video distribution amplifiers 
12-1 and 12-2 which are coupled to left and right video display 4L and 4R (13-1 
and 13-2 in Figure 2), and a video encoder 15. The cabinet may also house 
output only video/audio devices 16 (video laserdisc for example), video cassette 
recorders 17 for making a permanent record of the information presented to the 
audience or as a source of information to be merged by the system of this 
invention. Device 17 can also be an audio source. Cabinet 1 also houses the 
audio amplifier/speaker combination 18, and external connectivity jacks or 
connection panel 21 and internal connectivity 22 for the internal video/audio 
paths. Drop leafs 1DL and 1DR provide additional work space for the user. 

Video displays 4L and 4R (13-1 and 13-2 in Figure 2) for space reasons are 
preferably high quality flat panel displays and are mounted in panel TP which, 
for ergonomic reasons, can be swung forward on a hinge TPH and lockable in the 
adjusted position by a latch rod (not shown). Moreover, while displays 4L and 
4R are shown laterally aligned, they may be vertically aligned. In fact they need 
not be aligned and may be in individually adjustable mounts so the user can adjust 
them to best accommodate his or her comfort. 

As shown in Figure 1A the cabinet is positioned at any desired position by 
the user in relation to an audience 3, the members which of which are provided 
with data entry devices (not shown) to provide interactive responses to the 
material being presented to them. The speakers 8 are preferably facing the 
audience 3. Likewise the large area display screen 2 is positioned for easy 
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viewing by the audience. As discussed above, the large area display screen will 
have the same information displayed therein as one of screens 4. 

Control Integration and Computer Video Routing. The control 
system 9 manages the communication between and control of all devices within 
the system. Comprised of a microprocessor 24 that controls a wide variety of 
control and interface cards on a common bus 23. The bus architecture 23 need 
only handle data consisting of ASCII text strings and to address at least 255 
devices to ensure sufficient expendability to support a wide range of applications. 
Microprocessor 24 is programmable, has ROM to hold a minimal operating 
system, and has battery backed RAM to hold an executable program. The 
executable program runs in a continuous loop monitoring a register. The 
register holds ASCII commands strings sent by the attached computer platforms 
10-1 and 10-2. These command strings consist if an initial device code, a device 
number if more than one of a specific device exists, and a command. The 
executing program interprets this command and executes a subroutine 
appropriate for the device. A command structure has been developed that 
reduces the complexity of the code in computer platforms 10-1 and 10-2. For 
example, to play a video cassette, only VCR00PL is sent. The device is VCR, the 
particular VCR is 00 (indicating only one or the first VCR), and the command 
PL for play. PL is sent to play any type of device regardless of what is "plays". 
The subroutine that exists for VCR00 knows what commands to send over the 
RS-232 card (25-4 Figure 3) to get that device to play. Since many types and 
brands of devices have the same functionality the level of complexity is abstracted 
by using a lookup table for commands like PL (play) so that the computer 
platforms need only send PL and the control system does the lookup and control. 
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A plurality of RS-232 interface cards 25-1 to 25-5 are provided and support 
communication with computer platforms 10-1 and 10-2, video devices 16 and 17, 
audio devices, and switching matrices 19 and 20. Relay card 26 allows TTL 
voltage levels to be set to control external devices. Volume control card 27 takes 
in an audio voltage level and ramps it up or down at a selected rate via ASCII 
commands sent by the microprocessor 24. This allows a touch screen to control 
audio volume (see Figure 8). The computer platforms 10-1 and 10-2 are 
integrated into the system via RS-232 serial port connections 25-1 and 25-2. The 
computer platforms 10-1 and 10-2 support two displays, one for letting a 
presenter or multimedia author see the final result of the presentation and the 
other to provide the controls that manage the presentation being given or 
developed. Two input one output switch boxes 11-1, 11-2, and 11-3, under 
control of the relay card 26, allow a user via a touch screen to decide which 
computer platform controls the video displays 4L and 4R and video projector 14. 
This allows rapid access to large amounts of information in various formats via 
the touch screen. The video distribution amplifiers 12-1 and 12-2 are required to 
allows the analog computer video signal to drive both video display and video 
projector 14. It will be appreciated that digital displays can be incorporated in 
the invention. Video displays 13-1 and 13-2, which can be color multisync 
monitors, either CRT, liquid crystal or plasma, support the virtual interface for 
control of all components integrated by the system. Any video projection system 
14 or large screen display, can be used to get the desired information to a large 
audience. A video encoder 15 is used to convert computer video signals to a 
form that can be routed, transmitted, or stored to traditional video tape formats. 
Any video output device that can be controlled via infrared or RS-232 can be 
integrated into the system of this invention. Any amplified stereo speaker system 
18 can be controlled by the system of this invention. A video and audio switching 
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matrices 19 and 20 are used to route the various stereo and audio signals to and 
from the devices that need it. 

NTSC Video Out Routing (Figure 4). Each computer platform 
incorporates two video digitizers combined with two video display cards to 
display video in the computer video format on the two video displays 4L and 4R. 
Both displays require the video 30-1 and 30-2 be routed to them for digitization. 
This configuration supports each display to contain a different video source. Any 
device that can record video 31 can receive any available video source. Most 
video projectors handle both computer video signals as well as NTSC video which 
can both be routed to the projector. Providing a means to route video external 
33 to the system allows a user to create a presentation then get it to any device 
that can transmit, route, or store that video 31. 

NTSC Video In Routing (Figure 5). The video switching matrix 20 
can accept the video output (34, 35, 36, 37) of any device in the system. The 
encoded computer video signal 36 can be routed anywhere desired once put in the 
NTSC video format. Accepting external video signals 37 supports the addition of 
cameras, televideo conferencing systems, etc... 

Stereo Audio Out Routing (Figure 6). The computer platforms 10-1 
and 10-2 must be able to digitize stereo audio 38-1 and 38-2 for its integration 
into a presentation or multimedia database. Any device that can record audio can 
receive any available audio source. The volume control card 27 receives the 
desired audio signal 40-1, adjusts its volume by ramping up or down the voltage, 
them outputs that signal 40-2 to a fixed gain amplifier 1 8. Providing a means to 
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route audio external to the system allows a user to create a presentation then get it 
to any device that can transmit, route, or store that audio. 

Stereo Audio In Routing (Figure 7). The computer platforms 10-1 
and 10-2 output their own audio (internal speaker) 42-1 and 42-2, audio from 
internal CD players, and audio that is stored in digital format and converted to 
analog. The audio switching matrix 19 can accept the audio output of any device 
in the system. Accepting external audio signals 45 supports the addition of any 
device from radios to microphones that output an analog audio signal. 

Computer Platform Software (Figure 8-11). The software residing 
on computer platforms 10-1 and 10-2 has two integrated modules. The first is a 
virtual control interface (Figure 8) that allows generic control of all components 
within the system. The user can control a particular device by selecting its icon 
46 and controlling it with the various controls 49. If the device provides video it 
can be viewed 50. A user has full control over audio, both its volume 47 and its 
source 48. The user can play a VHS tape, play a CD, play a laserdisc, create a 
QuickTime movie, control a camera, change a cable TV channel, etc... Also a 
user can create an Edit Decision List (EDL) that stores from and to addresses 
(video/audio clip or sequence) from video tapes, laserdiscs, or CDs. The EDL 
for a video tape or other media allows a user to play, delete, record, digitize, or 
move clips or sequences of video and audio. 

The second module allows a user to rapidly create a multimedia presentation 
taking advantage of all of the components within the system. A slide show 
metaphor is used. A group of slides forms a presentation. An element of a 
presentation, the slide, contains backgrounds, titles, other forms of text, video, 
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audio, animations, charts, graphics, and interactive buttons. Figure 9 shows the 
presentation menu 51 and the ability to create, edit, delete, select, and rename a 
presentation. When a presentation is opened the slide menu (Figure 10) shows 
the interface for creating and editing slides. Slides are shown in miniature for 
ease of identification 57 and the slide to be edited or shown must be selected 58. 
A master slide is used when a slide element is added that is to be part of all slides 
56. Finally, the edit menu Figure 1 1 shows a slide being edited. This interface 
allows all the elements of the slide to manipulated as desired. This slide example 
shows and image imported from a Kodak Photo CD 61, a chart that was made 
from within the system 62, and the first frame of a video clip 63 which will 
become active when selected by the user. The slide is shown scaled down to 
allow the user access to controls, the slide can be shown full screen at any time 
desired. The system is designed for ease of use and does not require the user to 
program, write any scripts, or be familiar with any sequence of keyboard 
commands. 



While preferred embodiments of the invention have been shown and 
described, it will be appreciated that various modifications and adaptions of the 
invention will be obvious to those skilled in the art. What is claimed: 
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CLAIMS 

1 . A multimedia management system, comprising: 

a plurality of host computer platforms and operating system software 

therefor, 

two or more video display screens and means for connecting said 
video display screens to said host computer platforms, 

a plurality of media sources, including, video, audio, and 
communication sources, 

a pointing device including means providing horizontal and vertical 
coordinates to a selected computer platform, and 

a controllable interface means coupling said media sources to a 
selected host computer platform. 

2 . The multimedia management system defined in claim 1 including large 
screen display means and means for causing said- large display screen means to 
display the same information that is on a selected one of said two or more display 
screens. 

3 . The multimedia management system defined in claim 1 wherein said 
controllable interface means includes a programmable microprocessor and a bus 
system, a plurality of RS-232 interface cards connected to said bus system, said 
RS-232 interface cards being connected on a one-to-one basis with said plurality 
of host computer platforms and said plurality of media sources. 

4. The multimedia management system defined in claim 3 including a 
video switching matrix and an audio switching matrix for selectively coupling 
said plurality of media sources to a selected host computer. 
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5 . The multimedia management system defined in claim 1 wherein two of 
said two or more display screens are positioned for common viewing by a user of 
said system, and said pointing device is a touch panel device associated with one 
of said two display screens positioned for common viewing by said user. 

6 . The multimedia management system defined in claim 2 wherein said 
large screen display means is selected from a group comprising projection 
systems and a large area flat matrix display panels and is positioned for viewing 
by an audience. 

7. The multimedia management system defined in claim 6 including 
speaker means associated with said large screen display means. 

8. The multimedia management system defined in claim 7 wherein 
members of said audience are provided with a data entry device to record their 
responses to information presented on said large screen display and/or by said 
speaker means. 
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